Stereomicroscopes are conventionally used in fine processing under a microscope, or surgical operations under a microscope where, for example, accurate work on a small area around a lesion is required. Recently, there has been demand for conducting these tasks using remote control. If the capabilities of a remote operation are available, a processing engineer or a doctor can conduct the task from a remote location without traveling to the actual processing or operating site. It is desirable for realizing such a remote operation that images of an object observed by a stereomicroscope be formed and displayed on a display unit.
An apparatus that allows the viewer to three-dimensionally observe displayed images with the help of binocular parallax is known. For example, in some stereoscopic image observation apparatuses, images of an object are captured from different angles so that the effects of binocular parallax appear in images displayed on a display unit, and the viewer observes separate left and right images having parallax with his left and right eyes, respectively, for three-dimensional observation.
In such an apparatus, the resolution on the image pickup surface deteriorates as the optical system of the imaging part has a larger depth of field. On the other hand, as the optical system of the imaging part has a larger aperture for higher resolution on the image pickup surface, the depth of field inherently becomes smaller, and this can create problems. In applying a stereoscopic image observation apparatus to surgical operations and fine processing under a microscope, deterioration in resolution of observed images is not acceptable because it directly affects the accuracy of the operation performed by the operator. When such a stereoscopic image observation apparatus is used in a surgical operation under a microscope, the optical system of the imaging part inherently provides a smaller depth of field in order to obtain higher resolution observation images. Consequently, the operator is required to frequently refocus during the operation, causing lowered performance and operator fatigue.
It is known that stereoscopic images provided by the prior art stereoscopic image observation apparatuses are difficult to see three-dimensionally in the line of sight of the viewer. In other words, the larger features of an object image, for example, the general contour of an object, are relatively easy to see three-dimensionally. However, an object near the direction of the line of sight of the viewer is observed as lying in a plane with no three-dimensional appearance. Therefore, the viewer cannot recognize the object as being three-dimensional. Images lacking a three-dimensional appearance in the line of sight of the viewer may cause the operator to misunderstand the shape of the object and, therefore, are not suitable for the applications described above.
In order to solve the above problem, techniques using lenticular optical elements and holograms that can provide three-dimensional information in the line of sight of the viewer have been proposed in the prior art. However, these techniques do not provide an imaging system with sufficient resolution and it is difficult to put such techniques into practical use. Techniques using DFD (depth-fused 3D) devices have been proposed in Japanese Laid-Open Patent Application Nos. 2000-214413 and 2000-341473.
Japanese Laid-Open Patent Application No. 2000-214413 discloses that positional relationships among multiple images may be expressed by changing display densities of the same point of multiple images arranged in the line of sight. Japanese Laid-Open Patent Application No. 2000-341473 discloses a case in which a focused image and an unfocused image that are spaced from each other in the optical axis direction are separately captured and then displayed in an overlaid manner in order to increase the amount of information in the line of sight of an observer so that the observer recognizes a three-dimensional image.
It is generally considered in the prior art that an observer simply recognizes multiple images and never identifies a three-dimensional image when images are overlaid. Further, an unfocused image is considered to cause deterioration of an image, such as by reducing the contrast. Therefore, image correction such as deletion of unfocused areas is made.
However, in fact, when an unfocused image is overlaid in the line of sight of a viewer without changing the unfocused image, the unfocused image contributes to giving a three-dimensional appearance in the line of sight of the viewer, and thus the viewer can observe a natural three-dimensional image. The techniques described in Japanese Laid-Open Patent Application Nos. 2000-214413 and 2000-341473, described above, utilize the fact that the viewer recognizes multiple images in the line of sight as point information and considers changes in image contrast (i.e., changes in density of an image) to be three-dimensional information. However, three-dimensional images created by those techniques do not have a sufficient three-dimensional appearance for using them in surgical operations under a microscope.